feat(apim): Add implementation for azure apim#11743
Open
TophrC-dd wants to merge 36 commits into
Open
Conversation
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
…#11715) Include JVM distribution details in crash report Add runtime info to experimental instead Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
List iteration benchmark Merge branch 'master' into dougqh/list-iteration-benchmark Fixing silly oversight - commented out benchmarks to run one in isolation earlier A bit of clean-up Adding missing end ul to doc Moving iterator benchmarks next to enhancedFor Merge branch 'master' into dougqh/list-iteration-benchmark spotless Merge branch 'dougqh/list-iteration-benchmark' of github.com:DataDog/dd-trace-java into dougqh/list-iteration-benchmark Update internal-api/src/jmh/java/datadog/trace/util/ListIterationBenchmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com> Update internal-api/src/jmh/java/datadog/trace/util/ListIterationBenchmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com> Merge branch 'master' into dougqh/list-iteration-benchmark Update internal-api/src/jmh/java/datadog/trace/util/ListIterationBenchmark.java Co-authored-by: Sarah Chen <sarah.chen@datadoghq.com> Isolate per-thread collections in ListIterationBenchmark Build each thread's list (and its Elements) in a Scope.Thread @setup so the manipulate_* mutations stay thread-local. Previously the lists lived in enum constants shared across all 8 threads, so the benchmark measured cross-thread contention on Element.num rather than iteration cost. Also bump to @fork(2) and fix a Javadoc typo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Merge remote-tracking branch 'origin/master' into dougqh/list-iteration-benchmark Replace stale results in ListIterationBenchmark with Java 17 numbers Drop the old (pre-per-thread-state) results table; add a condensed Java 17 block. For ArrayList the direct styles (cstyleFor/forEach/enhanced-for/iterator) cluster within ~10%; stream() is ~3.6x slower; parallelStream() is catastrophic for small lists (ForkJoinPool overhead) and erratic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Merge branch 'master' into dougqh/list-iteration-benchmark Co-authored-by: sarahchen6 <sarah.chen@datadoghq.com> Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
init adding tests Merge branch 'master' into mhlidd/otlp_trace_metrics_config bugfix fixing issue where internal DD configs are used instead of OTel equivalents Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
Adding to StringUtiils - fast replaceAll for a fixed string & replacement, 3x throughput compared to regex based solutions, 1/2x allocation compared to regex solutions - added SubSequence which provides a view into a subsequence of a String without incurring extra allocation - Strings.spliit returns an Iterable<SubSequence> can be used to do light weight processing of a String Checking hashCode Reducing whitespace Fixing ugly spotless formatting Fixing ugly spotless formatting Renaming benchmark method Clarifying Javadoc grammar Allowing forbidden API usage These benchmarks exist to show why the APIs are forbidden Merge branch 'master' into dougqh/strings-improvements Merge branch 'master' into dougqh/strings-improvements Adding String.replace to the benchmark grammar Added comment explaining implementation approach Clean-up imports spotless Merge branch 'master' into dougqh/strings-improvements Fixing package after moving code from stand-alone reproduction Merge branch 'dougqh/strings-improvements' of github.com:DataDog/dd-trace-java into dougqh/strings-improvements Add motivation to SubSequence Javadoc (why avoid the substring allocation) Per bric3 review: the doc said what SubSequence avoids allocating but not why it matters. Explain the use case — allocation-free lightweight parsing: substring/ subSequence copy per call, so splitting a string into many pieces on a hot path allocates O(pieces) Strings; SubSequence is a zero-copy view. Also fixes a typo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Merge remote-tracking branch 'origin/master' into dougqh/strings-improvements Merge branch 'master' into dougqh/strings-improvements Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
Provide ability to capture continuations using the Context API This is meant as a stepping stone from the old API towards a simpler foundation Clean up use of EmptyContext.INSTANCE Apply @ParametersAreNonnullByDefault to test listeners Explain choice of rehash value Extend context/scope tests Introduce ContextHolder to hold/update current context without the overhead of ThreadLocal Replace weak-ref NoopContextScope cache with simple allocation Benchmarking shows these scopes aren't alive long enough to pay for the caching Greatly reduce allocations by treating existing context containers as no-op scopes Add microbenchmarks to exercise Context API for both old and new implementations Refactor no-op scope approach Allow contexts to wrap themselves as scopes without attaching. This helps reduce allocations when attaching the same context multiple times. Merge remote-tracking branch 'origin/master' into mcculls/context-continuations Rename previous to beforeAttach / beforeSwap Make DDSpan, NoopSpan (and BlackHoleSpan) all SelfScopedContexts to further reduce allocations Fix merge conflict Merge remote-tracking branch 'origin/master' into mcculls/context-continuations Use dotted property form for JMH options (jmh.includes) and testJvm property for the JMH JVM Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
fix: report correct block outcome tag in rasp.rule.match metric rasp.rule.match was always reporting block:N/A because raspRuleMatch() was called before the action loop; flow.isBlocking() is only valid after setAction() runs inside the loop. - Move raspRuleMatch() call to after the action loop (setRaspMatched stays in place) - Add boolean blocked param; double counter array for per-outcome tracking - Update prepareMetrics() to drain both blocked/non-blocked slots per RuleType - Add block: tag to both variant and non-variant RaspRuleMatch branches - Guard flow.setAction(rba) against null return from createBlockRequestAction/createRedirectRequestAction Merge branch 'master' into rasp-rule-match-fix Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
fix: avoid final field modification in Weaver instrumentation Update dd-java-agent/instrumentation/weaver-0.9/src/main/java/datadog/trace/instrumentation/weaver/WeaverInstrumentation.java Co-authored-by: Brice Dutheil <brice.dutheil@gmail.com> Update dd-java-agent/instrumentation/weaver-0.9/src/main/java/datadog/trace/instrumentation/weaver/WeaverInstrumentation.java Co-authored-by: Brice Dutheil <brice.dutheil@gmail.com> Co-authored-by: daniel.mohedano <daniel.mohedano@datadoghq.com>
chore: Move golden file and hard jar numbers to metadata Co-authored-by: brice.dutheil <brice.dutheil@datadoghq.com>
…pan index reference (#11739) feat(test-utils): Add PollingConditions test utility for async assertions feat(test-utils): Add childOfIndex matcher for span parent assertions fix(test-utils): Update span assertion error messages feat(test-utils): Add PollingConditions test utility for async assertions feat(test-utils): Add validation in PollingConditions Co-authored-by: bruce.bujon <bruce.bujon@datadoghq.com>
…ility-symbols) (#11614) feat(sca): migrate SCA Reachability to method-level symbol database format Updates the Gradle plugin and runtime transformer to consume the new sca-reachability-symbols/jvm database format, which ships method-level symbols (class + method name) instead of class-level symbols. Key changes: - GhsaEnrichmentParser: new target string format "package:ClassName.method"; GHSA ID read from vulnerability.id; one entry per dependency_name - ScaEnrichmentsPlugin: updated API URL to sca-reachability-symbols/jvm - ScaReachabilityTransformer: removed class-level detection (reportedHits, PendingClass, pendingClassEvents, processPendingClassEvents); simplified two-phase design to direct pendingRetransformNames enqueue on first load - ScaSymbol: method is now always non-null; isClassLevel() removed - ScaReachabilityHit: removed CLASS_LEVEL_SYMBOL constant - sca_cves.json: regenerated with junrar, zserio-runtime, tomcat-embed-core - New tests: ScaRealDatabaseTest, ScaRealLibraryBytecodeTest - Smoke test: migrated from snakeyaml to junrar 7.5.5 (GHSA-hf5p-q87m-crj7) tag: no release note Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols fix(sca): handle JARs without pom.properties in version resolution DependencyResolver.guessFallbackNoPom() parses the version from the filename when pom.properties is absent (e.g. junrar-7.5.5.jar). The resulting Dependency has name="junrar" (no group ID), which the previous exact-match logic could not match against the "com.github.junrar:junrar" artifact name. Add ScaReachabilityTransformer.matchVersion() as a shared helper that first tries an exact name match, then falls back to an artifact-ID-only match (dep.name without ":") for deps produced by guessFallbackNoPom. findArtifactInUrl() is updated to use the same helper so both resolution paths benefit. For the smoke test (shadow JAR): add a synthetic pom.properties for junrar under src/main/resources/META-INF/maven so DependencyResolver can find the version when all JARs are merged into one flat file. nit(sca): inline single-line findArtifactInUrl wrapper findArtifactInUrl was a private one-liner used in exactly one place. Inline it into findArtifactVersionInClasspath to remove unnecessary indirection. fix(sca): guard against null dep.name in matchVersion fallback guessFallbackNoPom can return a Dependency with name=null for JARs whose filename does not match the artifact-version pattern. The artifact-ID fallback loop called dep.name.contains(":") without a null check, which would throw NPE in that edge case. nit(sca): spotlessApply fix(sca): pre-warm jarCache before retransformClasses to avoid deadlock JAR I/O (resolveDependencies -> DependencyResolver -> JarReader) was happening inside the ClassFileTransformer callback, i.e. while the JVM holds its internal retransform locks. Libraries like snakeyaml that trigger class loading during JAR resolution can deadlock because the JVM class-loading lock and the retransform lock are both held simultaneously. Fix: in performPendingRetransforms(), resolve each class's JAR into jarCache on the telemetry thread BEFORE calling retransformClasses(). When the transform callback fires, resolveDependencies() returns immediately from cache with no I/O. This restores the v1 (PR #11352) invariant: JAR I/O on the telemetry thread only, bytecode injection only inside the callback. Expose jarCache and resolveDependencies as @VisibleForTesting to support the regression test. fix(sca): use cache-only lookup in processClass to prevent JAR I/O under JVM retransform locks processClass() runs inside the retransform callback while the JVM holds its internal retransform locks. Calling resolveDependencies() from there can deadlock (snakeyaml triggers class loading during JAR resolution, hitting the JVM class-loading lock while the retransform lock is already held). resolveDependenciesFromCache() only reads jarCache — no I/O — and returns empty on a cache miss. performPendingRetransforms() pre-warms jarCache for every class before calling retransformClasses(), so the cache is always populated when the callback fires. A cache miss (pre-warm failed) sets hasUnresolvedMethodLevelSymbols and re-queues the class for the next heartbeat. revert(sca): remove local-only spring-web CVE entry from sca_cves.json fix(sca): prevent hit re-emission when DependencyService re-detects the same JAR DependencyService can report the same JAR (artifact:version) on consecutive heartbeats when multiple classloaders in the application load it independently. Step 2 called peekSnapshot unconditionally whenever snapshotByKey had no entry, so a re-detection triggered peekSnapshot and re-emitted the existing hit to the backend on every heartbeat — a stale duplicate of data already received. Fix: track whether a dep is being seen for the first time via the return value of knownDeps.put(). Only call peekSnapshot and emit metadata:[] on the first detection (previous == null). Re-detections with no pending CVE state (no entry in snapshotByKey) produce no emission. A real new CVE state change (snapshotByKey has an entry from Step 1) is still emitted regardless — the previous == null guard only suppresses the peekSnapshot / metadata:[] fallback paths. Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols fix(sca): pre-warm classpathArtifactCache before retransformClasses to close aggregator-artifact deadlock gap The previous pre-warm loop only populated jarCache for each class's own JAR. For aggregator artifacts (e.g., spring-boot-starter-web) whose watched classes live in a different JAR, resolveVersionForArtifact() falls through to findArtifactVersionInClasspath(), which calls resolveDependencies() for every java.class.path entry — fresh JAR I/O under JVM retransform locks — reproducing the snakeyaml-style deadlock. Fix: extend the pre-warm loop to also call resolveVersionForArtifact() for each entry of each class being retransformed. This populates classpathArtifactCache on the telemetry thread before retransformClasses() acquires JVM locks, so the callback finds the version cached and never performs JAR I/O under locks. Adds a regression test using ObjectMapper (in jackson-databind.jar) with an entry whose artifact is jackson-core (a different JAR on the test classpath). Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols fix(sca): replace junrar:7.5.5 test dependency with minimal stubs Adds LocalFolderExtractor and FileHeader as test-only stubs under com.github.junrar to avoid shipping a test dependency with known CVEs. The tests only need the class/method signatures to exist on the classpath for ASM injection and reflection-based callback verification; the library's real implementation is not required. Addresses review comment from manuel-alvarez-alvarez on PR #11614. Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols fix(sca): replace zserio-runtime and tomcat-embed-core test deps with minimal stubs style(sca): add braces to braceless if-continue guards for consistency Merge branch 'master' into alejandro.gonzalez/sca-reachability-add-symbols fix(sca): use resolved dep.name for registerCve to prevent registry key mismatch for no-pom JARs For JARs without pom.properties, DependencyResolver.guessFallbackNoPom produces an artifactId-only name (e.g. "junrar" instead of "com.github.junrar:junrar"). The transformer was calling registerCve with entry.artifact() — the full groupId:artifactId from the DB — creating a registry key that mismatched the key DependencyService would later report. Result: the periodic action emitted two separate telemetry entries for the same physical JAR: - "junrar" with metadata:[] (source/hash present, CVE absent — Step 2 lookup missed) - "com.github.junrar:junrar" with CVE data but no source/hash (Step 3, knownDeps miss) Fix: resolveArtifactDep() returns the matched Dependency object (not just the version string), and processClass uses dep.name for registerCve and MethodCallbackSpec. This keeps the registry key consistent with what DependencyService will report regardless of pom.properties presence. Implementation: - Add matchDep() returning Dependency (matchVersion delegates to it) - Change classpathArtifactCache to cache Dependency (preserves resolved name) - Add resolveArtifactDep() returning Dependency; resolveVersionForArtifact delegates to it - Add findArtifactInClasspath() returning Dependency; findArtifactVersionInClasspath delegates to it The smoke test's synthetic pom.properties masked this bug by making both routes agree on the full groupId:artifactId name. No smoke test changes needed — the unit tests cover the fix: - ScaReachabilityTransformerJava9Test: resolveArtifactDep returns artifactId-only name for no-pom - ScaReachabilityPeriodicActionTest: CVE and source/hash merged into single entry for no-pom dep fix(sca): distinguish physical copies in ScaReachabilityPeriodicAction re-detection guard Replace the `previous == null` check in Step 2 with `isNewPhysicalCopy(previous, dep)`, which compares source path and content hash to differentiate a re-detection of the same physical JAR file from two distinct physical copies sharing the same name@version key. The old guard over-suppressed in Tomcat multi-webapp scenarios: two webapps shipping the same artifact under different WEB-INF/lib paths would share a single registry entry, so the second webapp's copy would never emit source/hash metadata to the backend. The new predicate emits when previous is null (first detection), or when source or hash differs (new physical copy). Re-detection of the exact same file (same source, same hash) is still suppressed to avoid re-emitting stale hits. Two regression tests added: - sameSourceHashReDetected_doesNotReEmit: re-detection of the same file produces no emission - differentSourceHashSameKey_emitsBothPhysicalCopies: two paths/hashes for the same name@version both emit independently Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
Fix javadoc warnings across modules Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Use JUnit example in ThreadUtils javadoc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Drop needless javadoc on Closure overload Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: alexey.kuznetsov <alexey.kuznetsov@datadoghq.com>
Fix buildSrc Gradle warnings Co-authored-by: alexey.kuznetsov <alexey.kuznetsov@datadoghq.com>
Bump and clean guava library usages. Merge branch 'master' into alexeyk/bump-and-clean-guava # Conflicts: # gradle/libs.versions.toml Fixed failed test that hard linked to old Guava. Drop unused guava. Use Java built-in charsets. Merge branch 'master' into alexeyk/bump-and-clean-guava Lock files updated. Merge branch 'master' into alexeyk/bump-and-clean-guava Merge branch 'master' into alexeyk/bump-and-clean-guava # Conflicts: # dd-java-agent/instrumentation/glassfish-3.0/gradle.lockfile # dd-java-agent/instrumentation/grizzly/grizzly-2.0/gradle.lockfile # dd-java-agent/instrumentation/java/java-concurrent/java-concurrent-1.8/gradle.lockfile # dd-java-agent/instrumentation/kafka/kafka-clients-0.11/gradle.lockfile # dd-java-agent/instrumentation/kafka/kafka-clients-3.8/gradle.lockfile # dd-java-agent/instrumentation/rs/jax-rs/jax-rs-client/jax-rs-client-2.0/gradle.lockfile Fixed lock file. Fixed review notes. Merge branch 'master' into alexeyk/bump-and-clean-guava Pin guava for old cassandra. Cleanup not needed guava pin for SofaRpc. Cleanup for cassandra. Merge branch 'master' into alexeyk/bump-and-clean-guava Co-authored-by: alexey.kuznetsov <alexey.kuznetsov@datadoghq.com>
Exclude ProfilerContext from coverage Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
…ment (#11734) Hoist containsTraceComment "<key>=" needles to constants SharedDBCommenter.containsTraceComment built nine "KEY + =" strings on every call: the keys are encode()-derived (not compile-time constants), so each "KEY + =" is a runtime concat. A non-matching comment ran all nine checks and allocated nine throwaway Strings per call. Precompute the needles once at class init as *_EQ constants; the contains-chain is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Add SharedDBCommenterBenchmark for containsTraceComment Throughput benchmark at @threads(8) over a realistic comment-content mix (mostly non-DD, the all-nine-checks worst case) to surface the per-call allocation removed by hoisting the "<key>=" needles to constants. Run the parent commit (baseline) vs this branch and read the ops/s delta; -prof gc (gc.alloc.rate.norm) corroborates the mechanism. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Refresh SharedDBCommenterBenchmark results to JDK 17 @fork(5) zulu-17 @fork(5), -prof gc: 33.5M -> 62.1M ops/s (~1.9x), 156 -> ~0 B/op. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Merge branch 'master' into dougqh/dbcommenter-alloc Merge branch 'master' into dougqh/dbcommenter-alloc Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
#11780) Add missing coverage tests for Matchers.AnyMatcher primitive overloads The existing parameterized tests always dispatch through matches(Object), leaving the 9 primitive overloads uncovered. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Split AnyMatcher overload tests into individual test cases Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Merge branch 'master' into dougqh/matchers-anymatcher-coverage-fix Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
Add missing coverage tests for Hashtable.D2 clear() had no test; remove() had no test for the not-found (null return) case. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Merge branch 'master' into dougqh/hashtable-d2-coverage-fix Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
…#11778) Add missing coverage tests for OptimizedTagMap and EntryReadingHelper Tests for stream(), compute(), computeIfAbsent(), and computeIfPresent() on OptimizedTagMap were absent; EntryReadingHelper had no test coverage at all since LegacyTagMap (its sole caller) was removed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Merge branch 'master' into dougqh/tagmap-coverage-fix Co-authored-by: devflow.devflow-routing-intake <devflow.devflow-routing-intake@kubernetes.us1.ddbuild.io>
Cover disabled ratelimited logging. Co-authored-by: alexey.kuznetsov <alexey.kuznetsov@datadoghq.com>
9ee9dc3 to
4fc0c29
Compare
Kafka / producer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
Kafka / consumer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
This PR introduces the api-management inferred proxy spans to Azure. It leverages the existing inferred proxy span component. This works by looking for a magic value in one of the received headers.

APMSVLS-539
In the headers we also add metadata for Http Method, Path, Region and origin domain name.
Motivation
My motivation for this is have the java-tracer have feature parity with Node, Python and .Net that already support this inferred span.
Additional Notes
For testing I have setup a multi service distributed trace with another service upstream from the Api Management span.